The present invention relates to a method and a test tool for verifying the functionality of a software based unit which is provided with an interface for its external communication.
When developing products, whether it is hardware or software products or a combination of the two, the products have to be thoroughly tested in order to verify their functionality prior to offering the products on the intended market.
Often, the verification, or testing, of a product has to be performed during all phases that are included in an entire product development cycle. Within the industries of telecommunications, computers and software, a product, or an element being part of an overall product, is most often characterised in that it includes one or more interfaces with which it communicates with an environment, the environment being external to the product or, in case of a product element, defined by the overall product. A significant part of the verification, or testing, of such a product or element, both which hereinafter will be referred to as a unit, constitutes so called xe2x80x9cblack boxxe2x80x9d testing. The use of the term black box for a unit indicates that there is no knowledge of the internal mechanisms of the unit. Even if some of the unit""s internal mechanisms or behaviour were known, it is in many cases more appropriate to view the unit as a black box when performing tests and verifying the functionality of the unit. When performing black box testing on a unit, specific input data to one or more of the interfaces of the unit will result in corresponding output data from the interfaces. One of the most commonly used methods for testing a software based unit is to view the unit as a black box and to verify its operation during a number of test cases.
The tests that are necessary to perform when verifying the functionality of a unit during black box testing are normally very time consuming. The personnel conducting the tests normally has to go through a number of different test cases. Each test case involves a high degree of manual interaction, especially during the phase of manual verification of output data, but also for generating the input data specific for the test case to be run.
Today there exists several tools for automated generation of input data, e.g., the test tool xe2x80x9cMEDUSAxe2x80x9d, which is a trademark of Microsoft Corporation. However, a major drawback which still exists, whether these existing test tools are used or not, is that the output data generated by a unit during black box testing still have to be verified manually. Moreover, when using tools which provide automated input generation the result is often huge amount of output data that need to be verified, a cumbersome task for the testing personnel.
The nature of the above mentioned test tools, i.e., automated data input generators, is that they create new sets of data for each run. One of the problems associated with verification of output data is the need for exact knowledge of the input data that corresponded to the output data. If something appears to be wrong in the output data, i.e., if the unit subject to test does not seem to function as expected, it must be possible to reproduce the subset of the input data that revealed the improper functioning of the unit. One obvious reason for this is the desire to use such identified and reproduced input data when repeating the same test case on the unit after the unit has been redesigned or reconfigured. Alternatively, it may be desired to perform the same test case on a second unit of the same kind as the first, in order to investigate whether or not the second unit shows the same improper functioning. Alternatively, it is simply desired to repeat a test case for a number of different units of the same type in order to verify, e.g., during production, that they all functions in accordance with the design.
The present invention overcomes at least some of the above mentioned problems associated with testing and verification of a product or a part thereof by providing a method and a test tool having the features as defined in the appended claims.
The invention is based on the idea of using pre-recorded data for the reproduction of a test case and for the verification of a unit subject to the test case. The pre-recorded data includes pre-recorded input data and pre-recorded output data. By analysing the pre-recorded input data and the pre-recorded output data, functional relationships are derived between them. These functional relationships are then included as functional output data in the pre-recorded output data. The pre-recorded input data is applied to an interface of the unit and the pre-recorded output data is compared with the data transmitted from the unit in response to the applied pre-recorded input data. If the data transmitted from the unit is in correspondence with the pre-recorded output data, the functionality of the unit in accordance with the specific test case has been verified.
The test case is something which covers a well-defined part of the unit""s functionality and intended operation. The kind of unit addressed is a unit which is implemented as software, or software in combination with hardware, and which provides one or more interfaces for interacting with its environment. Furthermore, the unit is of the kind which it is desired, or necessary, to test as a black box, i.e., to test and verify the unit by interacting with its interface(s) as if there was no or little knowledge of the internal behaviour of the unit. The reason for this being, e.g., that the internal behaviour of the unit is at least in part unknown or too complicated or too extensive to be described.
Thus, the present invention and the different aspects thereof concern the process of verifying the functionality of a software based unit which is provided with an interface for its external communication.
According to one aspect of the invention, there is provided a method which includes the steps of: a) placing an external test tool in operative communication with the software based unit by means of a packet data connection, wherein data applied to, and received from, said interface is included in data packets; b) recovering pre-recorded input data relating to a predefined test case and applying said input data on said interface; c) recovering pre-recorded output data having a verified relation with the pre-recorded input data; d) replacing at least part of said pre-recorded output data with functional relationships between said pre-recorded input data and said pre-recorded output data, obtaining a set of functional output data; and e) comparing the output data received from said packet data protocol interface with said set of functional output data.
According to another aspect of the invention, there is provided a test tool which includes: a) interface means for enabling operative communication with the software based unit by means of a packet data connection, wherein the communicated data is included in data packets; and b) processing means for: (i) reading pre-recorded input data relating to a predefined test case; (ii) applying the pre-recorded input on said software based unit via said interface means; (iii) reading pre-recorded output data relating to said predefined test case, the pre-recorded output data having a verified relation with the pre-recorded input data; (iv) replacing at least part of said pre-recorded output data with functional relationships between said pre-recorded input data and said pre-recorded output data, obtaining a set of functional output data; and (v) comparing the output data received from said software based unit via said interface means with said set of functional output data; and c) memory means for storing said functional relationships to be provided to said processing means.
Thus, according to the invention, an external test tool is operatively connected to the software based unit. The test tool is designed with interface means via which it is able to communicate with the software based unit and its interface over a data connection set up between the two interfaces. The test tool includes processing means executing computer instructions for controlling the operation of the test tool when verifying and communicating with the software based unit.
Within the scope of the present invention and the context of this description, a software based unit, or sometimes just unit, is to be interpreted as either a software product, a software component included by a software and/or hardware product, a hardware product including software, or even a complete system, such as a telecommunications or computer system, which includes software. Even if the software based unit is described as including one interface, it is to be understood that the unit may include several interfaces for its communication and, thus, its interaction with the test tool according to the invention. The input data to and output data from the software based unit is to be interpreted as a digitized information conveyed by a digital signal.
Tests of software based units can according to the invention be performed in different ways. The recording of input data to and output data from a unit is either done while the unit communicates with a test tool, or while the unit communicates in its normal operating environment, such as with an external unit which the test tool later will be simulating. In both cases the actual verification of a unit based on the recorded data is performed by the test tool. Furthermore, the recording may be performed on an already verified unit or it may involve a parallel manual verification of the unit. The recorded data is then used either for verifying the same unit after, e.g., a redesign or a reconfiguration, or, for verifying another unit of the same type as the unit for which the data was initially recorded. Thus, testing or development personnel can repeat the test cases for which input/output data have been recorded over and over again with a minimal amount of manual interaction.
If the software based unit subject to verification communicates with its environment by means of a packet data protocol, such as TCP/IP or X.25, the invention makes it feasible not only to verify such things as sequence numbering of the transferred data packets, but also any other information carried by the data packets.